---
title: "Configuring LLM Providers"
subtitle: "Use any LLM provider with the Opik Optimizer"
description: "Learn how to configure different LLM providers like OpenAI, Anthropic, Gemini, Azure, and Ollama for use with the Opik Agent Optimizer."
---

The Opik Agent Optimizer uses [LiteLLM](https://docs.litellm.ai/) under the hood, giving you access to 100+ LLM providers with a unified interface. This guide shows you how to configure different providers for both your **ChatPrompt** (the model that runs your prompt) and the **Optimizer** (the model that improves your prompt).

## Understanding the Two Model Types

When using the Opik Optimizer, there are two distinct models to configure:

| Model Type | Purpose | Recommendation |
|------------|---------|----------------|
| **ChatPrompt model** | The model that executes your prompt during evaluation | Use the same model as your production application |
| **Optimizer model** | The model that analyzes failures and generates improved prompts | Use the most capable model available for best results |

```python
from opik_optimizer import ChatPrompt, MetaPromptOptimizer

# ChatPrompt model - this is the model your prompt runs on
prompt = ChatPrompt(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "{question}"}
    ],
    model="gemini/gemini-2.0-flash"  # Your production model
)

# Optimizer model - this is the model that improves your prompt
optimizer = MetaPromptOptimizer(
    model="openai/gpt-4o"  # Use a powerful model for optimization
)
```

## LiteLLM Model Format

All models use the LiteLLM format: `provider/model-name`

```python
# Examples of the LiteLLM model format
model="openai/gpt-4o"                          # OpenAI
model="anthropic/claude-3-5-sonnet-20241022"   # Anthropic
model="gemini/gemini-2.0-flash"                # Google Gemini
model="azure/my-deployment-name"               # Azure OpenAI
model="ollama/llama3"                          # Ollama (local)
model="openrouter/google/gemini-2.0-flash"     # OpenRouter
```

## Provider Configuration

<Tabs>
  <Tab value="openai" title="OpenAI">
    ### OpenAI

    **Environment Variable:**
    ```bash
    export OPENAI_API_KEY="sk-..."
    ```

    **Available Models:**
    - `openai/gpt-4o` - Most capable model
    - `openai/gpt-4o-mini` - Fast and cost-effective
    - `openai/gpt-4-turbo` - Previous generation
    - `openai/o1` - Reasoning model
    - `openai/o3-mini` - Efficient reasoning model

    **Example:**
    ```python
    from opik_optimizer import ChatPrompt, MetaPromptOptimizer

    prompt = ChatPrompt(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "{question}"}
        ],
        model="openai/gpt-4o-mini"
    )

    optimizer = MetaPromptOptimizer(model="openai/gpt-4o")
    ```
  </Tab>

  <Tab value="anthropic" title="Anthropic">
    ### Anthropic (Claude)

    **Environment Variable:**
    ```bash
    export ANTHROPIC_API_KEY="sk-ant-..."
    ```

    **Available Models:**
    - `anthropic/claude-3-5-sonnet-20241022` - Best balance of speed and capability
    - `anthropic/claude-3-opus-20240229` - Most capable
    - `anthropic/claude-3-haiku-20240307` - Fastest

    **Example:**
    ```python
    from opik_optimizer import ChatPrompt, MetaPromptOptimizer

    prompt = ChatPrompt(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "{question}"}
        ],
        model="anthropic/claude-3-5-sonnet-20241022"
    )

    optimizer = MetaPromptOptimizer(model="anthropic/claude-3-5-sonnet-20241022")
    ```
  </Tab>

  <Tab value="gemini" title="Google Gemini">
    ### Google Gemini

    **Environment Variable:**
    ```bash
    export GOOGLE_API_KEY="..."
    # or
    export GEMINI_API_KEY="..."
    ```

    <Tip>
      Get your API key from [Google AI Studio](https://aistudio.google.com/apikey).
    </Tip>

    **Available Models:**
    - `gemini/gemini-2.0-flash` - Latest fast model
    - `gemini/gemini-1.5-pro` - Most capable
    - `gemini/gemini-1.5-flash` - Fast and efficient
    - `gemini/gemini-1.5-flash-8b` - Lightweight model

    **Example:**
    ```python
    from opik_optimizer import ChatPrompt, MetaPromptOptimizer

    prompt = ChatPrompt(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "{question}"}
        ],
        model="gemini/gemini-2.0-flash"
    )

    optimizer = MetaPromptOptimizer(model="gemini/gemini-1.5-pro")
    ```

    **Complete Gemini Example:**
    ```python
    import os
    from opik_optimizer import ChatPrompt, MetaPromptOptimizer
    from opik.evaluation.metrics import LevenshteinRatio
    import opik

    # Set your API key
    os.environ["GOOGLE_API_KEY"] = "your-api-key-here"

    # Initialize Opik client and create dataset
    client = opik.Opik()
    dataset = client.get_or_create_dataset(name="gemini-optimization-demo")
    dataset.insert([
        {"question": "What is machine learning?", "answer": "Machine learning is a subset of AI that enables systems to learn from data."},
        {"question": "What is Python?", "answer": "Python is a high-level programming language known for its readability."},
    ])

    # Define metric
    def answer_quality(dataset_item, llm_output):
        metric = LevenshteinRatio()
        return metric.score(reference=dataset_item["answer"], output=llm_output)

    # Configure prompt with Gemini
    prompt = ChatPrompt(
        messages=[
            {"role": "system", "content": "Answer the question concisely."},
            {"role": "user", "content": "{question}"}
        ],
        model="gemini/gemini-2.0-flash"
    )

    # Configure optimizer (can use same or different provider)
    optimizer = MetaPromptOptimizer(
        model="gemini/gemini-1.5-pro",
        n_threads=4
    )

    # Run optimization
    result = optimizer.optimize_prompt(
        prompt=prompt,
        dataset=dataset,
        metric=answer_quality,
        max_trials=5,
        n_samples=2
    )

    result.display()
    ```
  </Tab>

  <Tab value="azure" title="Azure OpenAI">
    ### Azure OpenAI

    **Environment Variables:**
    ```bash
    export AZURE_API_KEY="..."
    export AZURE_API_BASE="https://your-resource.openai.azure.com"
    export AZURE_API_VERSION="2024-02-15-preview"
    ```

    **Model Format:**
    ```python
    # Use your deployment name
    model="azure/your-deployment-name"
    ```

    **Example:**
    ```python
    from opik_optimizer import ChatPrompt, MetaPromptOptimizer

    prompt = ChatPrompt(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "{question}"}
        ],
        model="azure/gpt-4o-deployment"  # Your Azure deployment name
    )

    optimizer = MetaPromptOptimizer(model="azure/gpt-4o-deployment")
    ```
  </Tab>

  <Tab value="ollama" title="Ollama (Local)">
    ### Ollama (Local Models)

    **Setup:**
    1. Install Ollama from [ollama.ai](https://ollama.ai)
    2. Pull a model: `ollama pull llama3`
    3. Ollama runs on `http://localhost:11434` by default

    **Environment Variable (optional):**
    ```bash
    export OLLAMA_API_BASE="http://localhost:11434"
    ```

    **Available Models:**
    - `ollama/llama3` - Meta's Llama 3
    - `ollama/mistral` - Mistral 7B
    - `ollama/codellama` - Code-focused Llama
    - `ollama/phi3` - Microsoft Phi-3

    **Example:**
    ```python
    from opik_optimizer import ChatPrompt, MetaPromptOptimizer

    prompt = ChatPrompt(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "{question}"}
        ],
        model="ollama/llama3"
    )

    # Note: Local models may be slower for optimization
    optimizer = MetaPromptOptimizer(model="ollama/llama3")
    ```
  </Tab>

  <Tab value="openrouter" title="OpenRouter">
    ### OpenRouter

    OpenRouter provides access to multiple providers through a single API.

    **Environment Variable:**
    ```bash
    export OPENROUTER_API_KEY="sk-or-..."
    ```

    **Model Format:**
    ```python
    # Format: openrouter/provider/model
    model="openrouter/google/gemini-2.0-flash"
    model="openrouter/anthropic/claude-3-5-sonnet"
    model="openrouter/meta-llama/llama-3-70b-instruct"
    ```

    **Example:**
    ```python
    from opik_optimizer import ChatPrompt, MetaPromptOptimizer

    prompt = ChatPrompt(
        messages=[
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "{question}"}
        ],
        model="openrouter/google/gemini-2.0-flash"
    )

    optimizer = MetaPromptOptimizer(model="openrouter/anthropic/claude-3-5-sonnet")
    ```
  </Tab>
</Tabs>

## Environment Variables Reference

| Provider | Environment Variable | How to Get |
|----------|---------------------|------------|
| OpenAI | `OPENAI_API_KEY` | [platform.openai.com/api-keys](https://platform.openai.com/api-keys) |
| Anthropic | `ANTHROPIC_API_KEY` | [console.anthropic.com](https://console.anthropic.com/settings/keys) |
| Google Gemini | `GOOGLE_API_KEY` or `GEMINI_API_KEY` | [aistudio.google.com/apikey](https://aistudio.google.com/apikey) |
| Azure OpenAI | `AZURE_API_KEY`, `AZURE_API_BASE`, `AZURE_API_VERSION` | Azure Portal |
| OpenRouter | `OPENROUTER_API_KEY` | [openrouter.ai/keys](https://openrouter.ai/keys) |
| Ollama | None required (local) | [ollama.ai](https://ollama.ai) |

## Model Parameters

You can pass additional parameters to control model behavior:

```python
from opik_optimizer import ChatPrompt, MetaPromptOptimizer

# Configure model parameters for the ChatPrompt
prompt = ChatPrompt(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "{question}"}
    ],
    model="openai/gpt-4o-mini",
    model_parameters={
        "temperature": 0.7,
        "max_tokens": 1000,
        "top_p": 0.9
    }
)

# Configure model parameters for the Optimizer
optimizer = MetaPromptOptimizer(
    model="openai/gpt-4o",
    model_parameters={
        "temperature": 0.1,  # Lower temperature for more consistent optimization
        "max_tokens": 4096
    }
)
```

## Mixing Providers

You can use different providers for the ChatPrompt and Optimizer:

```python
from opik_optimizer import ChatPrompt, MetaPromptOptimizer

# Use a cost-effective model for prompt evaluation
prompt = ChatPrompt(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "{question}"}
    ],
    model="gemini/gemini-2.0-flash"  # Fast and affordable
)

# Use a powerful model for optimization reasoning
optimizer = MetaPromptOptimizer(
    model="openai/gpt-4o"  # Best reasoning capabilities
)
```

<Tip>
  **Recommendation:** Use a capable model like `gpt-4o` or `claude-3-5-sonnet` for the optimizer, even if your production application uses a smaller model. The optimizer only runs during development, so the cost is minimal compared to the quality improvements you'll achieve.
</Tip>

## Troubleshooting

<AccordionGroup>
  <Accordion title="AuthenticationError: Invalid API key">
    Ensure your API key is correctly set in the environment:
    ```bash
    # Check if the key is set
    echo $OPENAI_API_KEY
    
    # Set it if missing
    export OPENAI_API_KEY="sk-..."
    ```
  </Accordion>

  <Accordion title="Model not found">
    Verify the model name follows the LiteLLM format `provider/model-name`:
    ```python
    # ✅ Correct
    model="openai/gpt-4o"
    model="gemini/gemini-2.0-flash"
    
    # ❌ Incorrect
    model="gpt-4o"  # Missing provider prefix
    model="google/gemini-2.0-flash"  # Wrong provider name (use 'gemini')
    ```
  </Accordion>

  <Accordion title="Rate limiting errors">
    If you encounter rate limits, try:
    - Reducing `n_threads` in the optimizer
    - Using a model with higher rate limits
    - Adding delays between API calls
  </Accordion>
</AccordionGroup>

## Next Steps

- Learn about [Optimization Concepts](/agent_optimization/optimization/concepts)
- Explore different [Optimization Algorithms](/agent_optimization/algorithms/overview)
- Check out the [API Reference](/agent_optimization/advanced/api_reference)

<Info>
  For a complete list of supported providers and models, see the [LiteLLM documentation](https://docs.litellm.ai/docs/providers).
</Info>

